Pandas Profiling Report

Dataset statistics

Number of variables	27
Number of observations	827
Missing cells	74
Missing cells (%)	0.3%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	174.6 KiB
Average record size in memory	216.2 B

Variable types

NUM	19
CAT	8

Warnings

`name` has a high cardinality: 195 distinct values	High cardinality
`saifi_nomed` has a high cardinality: 472 distinct values	High cardinality
`caidi_nomed` has a high cardinality: 791 distinct values	High cardinality
`gen_mwh` has a high cardinality: 402 distinct values	High cardinality
`dem_res_mwh` has a high cardinality: 334 distinct values	High cardinality
`total_cust` is highly correlated with `total_mwh`	High correlation
`total_mwh` is highly correlated with `total_cust`	High correlation
`nm_mwh` is highly correlated with `pv_mwh`	High correlation
`pv_mwh` is highly correlated with `nm_mwh`	High correlation
`nm_pct` is highly correlated with `pv_pct`	High correlation
`pv_pct` is highly correlated with `nm_pct`	High correlation
`voltage` has 73 (8.8%) missing values	Missing
`wind_mwh` is highly skewed (γ1 = 20.1350571)	Skewed
`purchase_mwh` has unique values	Unique
`voltage` has 378 (45.7%) zeros	Zeros
`pv_mwh` has 400 (48.4%) zeros	Zeros
`wind_mwh` has 609 (73.6%) zeros	Zeros
`nm_mwh` has 396 (47.9%) zeros	Zeros
`ee_mwh` has 35 (4.2%) zeros	Zeros
`dem_res_customers` has 24 (2.9%) zeros	Zeros
`pv_pct` has 400 (48.4%) zeros	Zeros
`wind_pct` has 609 (73.6%) zeros	Zeros
`nm_pct` has 396 (47.9%) zeros	Zeros
`ee_pct` has 35 (4.2%) zeros	Zeros
`dem_res_cust_pct` has 25 (3.0%) zeros	Zeros
`circuit_pct` has 451 (54.5%) zeros	Zeros

Reproduction

Analysis started	2020-12-18 04:57:11.257485
Analysis finished	2020-12-18 04:58:42.162922
Duration	1 minute and 30.91 seconds
Software version	pandas-profiling v2.9.0
Download configuration	config.yaml

year
Real number (ℝ_≥0)

Distinct	7
Distinct (%)	0.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	2016.131802
Minimum	2013
Maximum	2019
Zeros	0
Zeros (%)	0.0%
Memory size	6.5 KiB

Quantile statistics

Minimum	2013
5-th percentile	2013
Q1	2014
median	2016
Q3	2018
95-th percentile	2019
Maximum	2019
Range	6
Interquartile range (IQR)	4

Descriptive statistics

Standard deviation	1.99230773
Coefficient of variation (CV)	0.0009881832767
Kurtosis	-1.25966747
Mean	2016.131802
Median Absolute Deviation (MAD)	2
Skewness	-0.05083934622
Sum	1667341
Variance	3.969290091
Monotocity	Decreasing

Histogram with fixed size bins (bins=7)

Value	Count	Frequency (%)
2019	133	16.1%
2014	128	15.5%
2018	123	14.9%
2016	119	14.4%
2017	118	14.3%
2015	110	13.3%
2013	96	11.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2013	96	11.6%
2014	128	15.5%
2015	110	13.3%
2016	119	14.4%
2017	118	14.3%

Value	Count	Frequency (%)
2019	133	16.1%
2018	123	14.9%
2017	118	14.3%
2016	119	14.4%
2015	110	13.3%

name
Categorical

HIGH CARDINALITY

Distinct	195
Distinct (%)	23.6%
Missing	0
Missing (%)	0.0%
Memory size	6.5 KiB

Tri-County Electric Coop	9
San Diego Gas & Electric Co	7
PECO Energy Co	7
North Arkansas Elec Coop, Inc	7
Ohio Edison Co	7
Other values (190)	790

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
Tri-County Electric Coop	9	1.1%
San Diego Gas & Electric Co	7	0.8%
PECO Energy Co	7	0.8%
North Arkansas Elec Coop, Inc	7	0.8%
Ohio Edison Co	7	0.8%
Southern California Edison Co	7	0.8%
Sierra Pacific Power Co	7	0.8%
Louisville Gas & Electric Co	7	0.8%
Lyon Rural Electric Coop	7	0.8%
Northern States Power Co - Minnesota	7	0.8%
Other values (185)	755	91.3%

Frequencies of value counts

Unique

Unique	40 ?
Unique (%)	4.8%

Histogram of lengths of the category

Length

Max length	39
Median length	26
Mean length	25.70858525
Min length	12

state
Categorical

Distinct	41
Distinct (%)	5.0%
Missing	0
Missing (%)	0.0%
Memory size	6.5 KiB

MN	85
WI	51
NC	49
IN	42
SC	41
Other values (36)	559

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
MN	85	10.3%
WI	51	6.2%
NC	49	5.9%
IN	42	5.1%
SC	41	5.0%
TX	41	5.0%
AZ	39	4.7%
CA	38	4.6%
OH	35	4.2%
GA	34	4.1%
Other values (31)	372	45.0%

Frequencies of value counts

Unique

Unique	2 ?
Unique (%)	0.2%

Histogram of lengths of the category

Length

Max length	2
Median length	2
Mean length	2
Min length	2

total_mwh
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	826
Distinct (%)	99.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	9345694.03
Minimum	57432
Maximum	111955723
Zeros	0
Zeros (%)	0.0%
Memory size	6.5 KiB

Quantile statistics

Minimum	57432
5-th percentile	135136.2
Q1	502877.5
median	2327987
Q3	12053663
95-th percentile	36982188.2
Maximum	111955723
Range	111898291
Interquartile range (IQR)	11550785.5

Descriptive statistics

Standard deviation	15836364.06
Coefficient of variation (CV)	1.694509152
Kurtosis	13.56327341
Mean	9345694.03
Median Absolute Deviation (MAD)	2104514
Skewness	3.251402801
Sum	7728888963
Variance	2.507904267e+14
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
345269	2	0.2%
18284543	1	0.1%
73554	1	0.1%
8432160	1	0.1%
59434	1	0.1%
1070398	1	0.1%
197482	1	0.1%
355662	1	0.1%
727706	1	0.1%
247462	1	0.1%
Other values (816)	816	98.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
57432	1	0.1%
59151	1	0.1%
59380	1	0.1%
59434	1	0.1%
60125	1	0.1%

Value	Count	Frequency (%)
111955723	1	0.1%
110072760	1	0.1%
109449144	1	0.1%
108513594	1	0.1%
104431096	1	0.1%

total_cust
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	824
Distinct (%)	99.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	465552.5139
Minimum	2109
Maximum	5236724
Zeros	0
Zeros (%)	0.0%
Memory size	6.5 KiB

Quantile statistics

Minimum	2109
5-th percentile	6722.5
Q1	24274
median	95602
Q3	512497.5
95-th percentile	1865335.1
Maximum	5236724
Range	5234615
Interquartile range (IQR)	488223.5

Descriptive statistics

Standard deviation	851224.1907
Coefficient of variation (CV)	1.828417129
Kurtosis	14.4043783
Mean	465552.5139
Median Absolute Deviation (MAD)	85649
Skewness	3.490218705
Sum	385011929
Variance	7.245826228e+11
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1303473	3	0.4%
145033	2	0.2%
74751	1	0.1%
157211	1	0.1%
151722	1	0.1%
210089	1	0.1%
6334	1	0.1%
56234	1	0.1%
701085	1	0.1%
2730588	1	0.1%
Other values (814)	814	98.4%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2109	1	0.1%
2132	1	0.1%
2163	1	0.1%
2189	1	0.1%
2211	1	0.1%

Value	Count	Frequency (%)
5236724	1	0.1%
5188308	1	0.1%
5069189	1	0.1%
5061483	1	0.1%
5020876	1	0.1%

no
Real number (ℝ_≥0)

Distinct	183
Distinct (%)	22.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	12348.36155
Minimum	329
Maximum	55937
Zeros	0
Zeros (%)	0.0%
Memory size	6.5 KiB

Quantile statistics

Minimum	329
5-th percentile	1015
Q1	5780.5
median	13664
Q3	16945.5
95-th percentile	20885
Maximum	55937
Range	55608
Interquartile range (IQR)	11165

Descriptive statistics

Standard deviation	7549.575265
Coefficient of variation (CV)	0.6113827519
Kurtosis	7.563501311
Mean	12348.36155
Median Absolute Deviation (MAD)	4616
Skewness	1.412761106
Sum	10212095
Variance	56996086.68
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
17166	7	0.8%
19490	7	0.8%
803	7	0.8%
11298	7	0.8%
19981	7	0.8%
9191	7	0.8%
14328	7	0.8%
13676	7	0.8%
11249	7	0.8%
14940	7	0.8%
Other values (173)	757	91.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
329	6	0.7%
407	1	0.1%
554	1	0.1%
590	7	0.8%
636	1	0.1%

Value	Count	Frequency (%)
55937	7	0.8%
26939	3	0.4%
25295	5	0.6%
24949	7	0.8%
24211	6	0.7%

type
Categorical

Distinct	5
Distinct (%)	0.6%
Missing	0
Missing (%)	0.0%
Memory size	6.5 KiB

Cooperative	362
Investor Owned	335
Municipal	99
Political Subdivision	24
State	7

Frequencies
Length

Value	Count	Frequency (%)
Cooperative	362	43.8%
Investor Owned	335	40.5%
Municipal	99	12.0%
Political Subdivision	24	2.9%
State	7	0.8%

Frequencies of value counts

Unique

Unique	0 ?
Unique (%)	0.0%

Histogram of lengths of the category

Length

Max length	21
Median length	11
Mean length	12.21523579
Min length	5

saidi_nomed
Real number (ℝ_≥0)

Distinct	766
Distinct (%)	92.6%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	104.247237
Minimum	0
Maximum	812.9
Zeros	1
Zeros (%)	0.1%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	23.605
Q1	59.49
median	89.01
Q3	133.65
95-th percentile	218.9342
Maximum	812.9
Range	812.9
Interquartile range (IQR)	74.16

Descriptive statistics

Standard deviation	73.23225751
Coefficient of variation (CV)	0.7024863163
Kurtosis	16.73926098
Mean	104.247237
Median Absolute Deviation (MAD)	34.15
Skewness	2.850259763
Sum	86212.465
Variance	5362.963539
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
143	5	0.6%
81	4	0.5%
138	4	0.5%
93	3	0.4%
58.8	3	0.4%
71	3	0.4%
95	3	0.4%
106	3	0.4%
158.4	3	0.4%
94	3	0.4%
Other values (756)	793	95.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	1	0.1%
3.023	1	0.1%
4.445	1	0.1%
4.83	1	0.1%
6.3	1	0.1%

Value	Count	Frequency (%)
812.9	1	0.1%
599.765	1	0.1%
574.822	1	0.1%
490.686	1	0.1%
421.4	1	0.1%

saifi_nomed
Categorical

HIGH CARDINALITY

Distinct	472
Distinct (%)	57.1%
Missing	0
Missing (%)	0.0%
Memory size	6.5 KiB

.	30
1	11
0.87	11
1.17	7
0.58	7
Other values (467)	761

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
.	30	3.6%
1	11	1.3%
0.87	11	1.3%
1.17	7	0.8%
0.58	7	0.8%
1.06	7	0.8%
0.79	6	0.7%
0.82	6	0.7%
0.66	6	0.7%
0.9	6	0.7%
Other values (462)	730	88.3%

Frequencies of value counts

Unique

Unique	333 ?
Unique (%)	40.3%

Histogram of lengths of the category

Length

Max length	5
Median length	4
Mean length	4.188633615
Min length	1

caidi_nomed
Categorical

HIGH CARDINALITY

Distinct	791
Distinct (%)	95.6%
Missing	0
Missing (%)	0.0%
Memory size	6.5 KiB

.	31
100	4
44.155844	2
64	2
144.44444	2
Other values (786)	786

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
.	31	3.7%
100	4	0.5%
44.155844	2	0.2%
64	2	0.2%
144.44444	2	0.2%
103.14465	1	0.1%
112.57576	1	0.1%
92.5	1	0.1%
74.141176	1	0.1%
92.747253	1	0.1%
Other values (781)	781	94.4%

Frequencies of value counts

Unique

Unique	786 ?
Unique (%)	95.0%

Histogram of lengths of the category

Length

Max length	9
Median length	9
Mean length	8.269649335
Min length	1

circuits
Real number (ℝ_≥0)

Distinct	428
Distinct (%)	51.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	579.2478839
Minimum	5
Maximum	5619
Zeros	0
Zeros (%)	0.0%
Memory size	6.5 KiB

Quantile statistics

Minimum	5
5-th percentile	26
Q1	65
median	178
Q3	759
95-th percentile	2334.1
Maximum	5619
Range	5614
Interquartile range (IQR)	694

Descriptive statistics

Standard deviation	922.2076039
Coefficient of variation (CV)	1.59207764
Kurtosis	9.403079463
Mean	579.2478839
Median Absolute Deviation (MAD)	140
Skewness	2.83717254
Sum	479038
Variance	850466.8646
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
87	12	1.5%
27	12	1.5%
40	12	1.5%
68	11	1.3%
36	11	1.3%
64	9	1.1%
20	9	1.1%
60	9	1.1%
74	8	1.0%
62	8	1.0%
Other values (418)	726	87.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
5	6	0.7%
7	1	0.1%
11	1	0.1%
16	1	0.1%
18	1	0.1%

Value	Count	Frequency (%)
5619	1	0.1%
5596	1	0.1%
5525	2	0.2%
5484	2	0.2%
5464	1	0.1%

voltage
Real number (ℝ_≥0)

MISSING
ZEROS

Distinct	182
Distinct (%)	24.1%
Missing	73
Missing (%)	8.8%
Infinite	0
Infinite (%)	0.0%

Mean	158.5
Minimum	0
Maximum	2869
Zeros	378
Zeros (%)	45.7%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	73
95-th percentile	1029.05
Maximum	2869
Range	2869
Interquartile range (IQR)	73

Descriptive statistics

Standard deviation	422.4063622
Coefficient of variation (CV)	2.665024367
Kurtosis	17.22259943
Mean	158.5
Median Absolute Deviation (MAD)	0
Skewness	3.934946187
Sum	119509
Variance	178427.1348
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	378	45.7%
1	13	1.6%
44	12	1.5%
10	11	1.3%
13	10	1.2%
11	9	1.1%
32	9	1.1%
12	8	1.0%
6	7	0.8%
4	7	0.8%
Other values (172)	290	35.1%
(Missing)	73	8.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	378	45.7%
1	13	1.6%
2	1	0.1%
3	4	0.5%
4	7	0.8%

Value	Count	Frequency (%)
2869	1	0.1%
2787	1	0.1%
2786	1	0.1%
2761	1	0.1%
2740	1	0.1%

nerc
Categorical

Distinct	10
Distinct (%)	1.2%
Missing	0
Missing (%)	0.0%
Memory size	6.5 KiB

MRO	198
SERC	183
RFC	159
WECC	144
SPP	56
Other values (5)	87

Frequencies
Length

Value	Count	Frequency (%)
MRO	198	23.9%
SERC	183	22.1%
RFC	159	19.2%
WECC	144	17.4%
SPP	56	6.8%
NPCC	29	3.5%
TRE	29	3.5%
FRCC	27	3.3%
MISO	1	0.1%
HI	1	0.1%

Frequencies of value counts

Unique

Unique	2 ?
Unique (%)	0.2%

Histogram of lengths of the category

Length

Max length	4
Median length	3
Mean length	3.46311971
Min length	2

gen_mwh
Categorical

HIGH CARDINALITY

Distinct	402
Distinct (%)	48.6%
Missing	0
Missing (%)	0.0%
Memory size	6.5 KiB

.	329
0.0	68
0	30
59	2
10615897	1
Other values (397)	397

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
.	329	39.8%
0.0	68	8.2%
0	30	3.6%
59	2	0.2%
10615897	1	0.1%
3692211	1	0.1%
15175161	1	0.1%
10817321	1	0.1%
15219200	1	0.1%
13363817	1	0.1%
Other values (392)	392	47.4%

Frequencies of value counts

Unique

Unique	398 ?
Unique (%)	48.1%

Histogram of lengths of the category

Length

Max length	11
Median length	3
Mean length	4.160822249
Min length	1

purchase_mwh
Real number (ℝ_≥0)

UNIQUE

Distinct	827
Distinct (%)	100.0%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	4911936.447
Minimum	61207
Maximum	96778091
Zeros	0
Zeros (%)	0.0%
Memory size	6.5 KiB

Quantile statistics

Minimum	61207
5-th percentile	149341.2
Q1	514131
median	1565619
Q3	5369209
95-th percentile	19402625.8
Maximum	96778091
Range	96716884
Interquartile range (IQR)	4855078

Descriptive statistics

Standard deviation	9119144.441
Coefficient of variation (CV)	1.856527367
Kurtosis	31.12538555
Mean	4911936.447
Median Absolute Deviation (MAD)	1328027
Skewness	4.801381023
Sum	4062171442
Variance	8.315879533e+13
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1678410	1	0.1%
3050197	1	0.1%
345444	1	0.1%
2329366	1	0.1%
611012	1	0.1%
217956	1	0.1%
840386	1	0.1%
4314638	1	0.1%
149680	1	0.1%
7099894	1	0.1%
Other values (817)	817	98.8%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
61207	1	0.1%
61571	1	0.1%
62658	1	0.1%
62671	1	0.1%
62802	1	0.1%

Value	Count	Frequency (%)
96778091	1	0.1%
72293537	1	0.1%
70487195	1	0.1%
70078621	1	0.1%
69376696	1	0.1%

pv_mwh
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	423
Distinct (%)	51.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	2691.103571
Minimum	0
Maximum	174137.899
Zeros	400
Zeros (%)	48.4%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0.838
Q3	175.45
95-th percentile	10182.2472
Maximum	174137.899
Range	174137.899
Interquartile range (IQR)	175.45

Descriptive statistics

Standard deviation	14105.40098
Coefficient of variation (CV)	5.24149317
Kurtosis	76.14927457
Mean	2691.103571
Median Absolute Deviation (MAD)	0.838
Skewness	8.142343136
Sum	2225542.653
Variance	198962336.9
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	400	48.4%
177	3	0.4%
181	2	0.2%
46	2	0.2%
0.068	2	0.2%
107.29	1	0.1%
420.68	1	0.1%
229.71	1	0.1%
17.139	1	0.1%
1046.17	1	0.1%
Other values (413)	413	49.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	400	48.4%
0.002	1	0.1%
0.013	1	0.1%
0.032	1	0.1%
0.038	1	0.1%

Value	Count	Frequency (%)
174137.899	1	0.1%
169882.509	1	0.1%
139279.701	1	0.1%
114205.756	1	0.1%
100497.167	1	0.1%

wind_mwh
Real number (ℝ_≥0)

SKEWED
ZEROS

Distinct	208
Distinct (%)	25.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	38.81987062
Minimum	0
Maximum	11648.213
Zeros	609
Zeros (%)	73.6%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	0.016
95-th percentile	67.9541
Maximum	11648.213
Range	11648.213
Interquartile range (IQR)	0.016

Descriptive statistics

Standard deviation	526.8190703
Coefficient of variation (CV)	13.57086106
Kurtosis	413.9171896
Mean	38.81987062
Median Absolute Deviation (MAD)	0
Skewness	20.1350571
Sum	32104.033
Variance	277538.3329
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	609	73.6%
0.08	4	0.5%
0.362	3	0.4%
1	3	0.4%
0.004	2	0.2%
0.028	2	0.2%
0.009	2	0.2%
0.724	2	0.2%
1.299	1	0.1%
192.767	1	0.1%
Other values (198)	198	23.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	609	73.6%
0.004	2	0.2%
0.005	1	0.1%
0.006	1	0.1%
0.007	1	0.1%

Value	Count	Frequency (%)
11648.213	1	0.1%
9545.512	1	0.1%
1145.001	1	0.1%
586.328	1	0.1%
574.553	1	0.1%

nm_mwh
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	426
Distinct (%)	51.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	2736.156778
Minimum	0
Maximum	174137.899
Zeros	396
Zeros (%)	47.9%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	1.855
Q3	209.973
95-th percentile	10461.6106
Maximum	174137.899
Range	174137.899
Interquartile range (IQR)	209.973

Descriptive statistics

Standard deviation	14151.07547
Coefficient of variation (CV)	5.171880348
Kurtosis	75.08903264
Mean	2736.156778
Median Absolute Deviation (MAD)	1.855
Skewness	8.072683917
Sum	2262801.655
Variance	200252936.9
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	396	47.9%
177	3	0.4%
141	2	0.2%
16	2	0.2%
46	2	0.2%
0.068	2	0.2%
14.626	1	0.1%
92.018	1	0.1%
1522.754	1	0.1%
41300	1	0.1%
Other values (416)	416	50.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	396	47.9%
0.002	1	0.1%
0.013	1	0.1%
0.038	1	0.1%
0.039	1	0.1%

Value	Count	Frequency (%)
174137.899	1	0.1%
169882.509	1	0.1%
139279.701	1	0.1%
114205.756	1	0.1%
100497.247	1	0.1%

ee_mwh
Real number (ℝ_≥0)

ZEROS

Distinct	767
Distinct (%)	92.7%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	118320.0378
Minimum	0
Maximum	2865573
Zeros	35
Zeros (%)	4.2%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	4.03
Q1	589.4
median	5739
Q3	119200
95-th percentile	541983.2
Maximum	2865573
Range	2865573
Interquartile range (IQR)	118610.6

Descriptive statistics

Standard deviation	266238.8467
Coefficient of variation (CV)	2.250158566
Kurtosis	29.20703023
Mean	118320.0378
Median Absolute Deviation (MAD)	5734.9
Skewness	4.592582904
Sum	97850671.27
Variance	7.088312347e+10
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	35	4.2%
38	5	0.6%
80	3	0.4%
5	3	0.4%
65	3	0.4%
509	2	0.2%
63	2	0.2%
122	2	0.2%
2613	2	0.2%
969	2	0.2%
Other values (757)	768	92.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	35	4.2%
0.002	1	0.1%
0.08	1	0.1%
1.5	1	0.1%
1.739	1	0.1%

Value	Count	Frequency (%)
2865573	1	0.1%
2180044	1	0.1%
2097423	1	0.1%
1747609	1	0.1%
1655101	1	0.1%

dem_res_customers
Real number (ℝ_≥0)

ZEROS

Distinct	662
Distinct (%)	80.1%
Missing	1
Missing (%)	0.1%
Infinite	0
Infinite (%)	0.0%

Mean	53382.84262
Minimum	0
Maximum	1323922
Zeros	24
Zeros (%)	2.9%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	2
Q1	222
median	3807.5
Q3	23984
95-th percentile	378852.75
Maximum	1323922
Range	1323922
Interquartile range (IQR)	23762

Descriptive statistics

Standard deviation	158589.3593
Coefficient of variation (CV)	2.970792703
Kurtosis	24.53806131
Mean	53382.84262
Median Absolute Deviation (MAD)	3798.5
Skewness	4.653784698
Sum	44094228
Variance	2.515058489e+10
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	24	2.9%
2	17	2.1%
1	15	1.8%
4	13	1.6%
3	10	1.2%
20	7	0.8%
8	7	0.8%
14	6	0.7%
11	5	0.6%
7	5	0.6%
Other values (652)	717	86.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	24	2.9%
1	15	1.8%
2	17	2.1%
3	10	1.2%
4	13	1.6%

Value	Count	Frequency (%)
1323922	1	0.1%
1246650	1	0.1%
1222371	1	0.1%
1178566	1	0.1%
1133468	1	0.1%

dem_res_mwh
Categorical

HIGH CARDINALITY

Distinct	334
Distinct (%)	40.4%
Missing	0
Missing (%)	0.0%
Memory size	6.5 KiB

.	177
0	159
0.0	68
	28
1	11
Other values (329)	384

Frequencies
Length

Common Values
Overview

Value	Count	Frequency (%)
.	177	21.4%
0	159	19.2%
0.0	68	8.2%
	28	3.4%
1	11	1.3%
5	5	0.6%
62	4	0.5%
20	4	0.5%
47	4	0.5%
3	3	0.4%
Other values (324)	364	44.0%

Frequencies of value counts

Unique

Unique	293 ?
Unique (%)	35.4%

Histogram of lengths of the category

Length

Max length	8
Median length	2
Mean length	2.347037485
Min length	1

pv_pct
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	428
Distinct (%)	51.8%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.05855384643
Minimum	0
Maximum	2.737363978
Zeros	400
Zeros (%)	48.4%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	2.329365372e-05
Q3	0.01163025297
95-th percentile	0.2473360579
Maximum	2.737363978
Range	2.737363978
Interquartile range (IQR)	0.01163025297

Descriptive statistics

Standard deviation	0.2444340285
Coefficient of variation (CV)	4.17451702
Kurtosis	65.28739354
Mean	0.05855384643
Median Absolute Deviation (MAD)	2.329365372e-05
Skewness	7.458000387
Sum	48.424031
Variance	0.05974799428
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	400	48.4%
0.006627365696	1	0.1%
0.2670386381	1	0.1%
0.1069705023	1	0.1%
0.03152009524	1	0.1%
0.08837235494	1	0.1%
0.02537508511	1	0.1%
0.0263774484	1	0.1%
0.003791046148	1	0.1%
0.0009563861411	1	0.1%
Other values (418)	418	50.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	400	48.4%
3.16228032e-07	1	0.1%
2.673417388e-06	1	0.1%
3.142061705e-06	1	0.1%
5.819624304e-06	1	0.1%

Value	Count	Frequency (%)
2.737363978	1	0.1%
2.648360265	1	0.1%
2.646364893	1	0.1%
2.163690212	1	0.1%
1.956577907	1	0.1%

wind_pct
Real number (ℝ_≥0)

ZEROS

Distinct	219
Distinct (%)	26.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.002713544559
Minimum	0
Maximum	0.1141962336
Zeros	609
Zeros (%)	73.6%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	4.756518746e-07
95-th percentile	0.009655891741
Maximum	0.1141962336
Range	0.1141962336
Interquartile range (IQR)	4.756518746e-07

Descriptive statistics

Standard deviation	0.01261911198
Coefficient of variation (CV)	4.650416349
Kurtosis	37.96209193
Mean	0.002713544559
Median Absolute Deviation (MAD)	0
Skewness	5.927243924
Sum	2.24410135
Variance	0.0001592419872
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	609	73.6%
0.0002968532956	1	0.1%
5.580210277e-05	1	0.1%
0.001046815935	1	0.1%
0.0004928349771	1	0.1%
0.02451280794	1	0.1%
0.03240393135	1	0.1%
0.0001674066039	1	0.1%
0.005406847083	1	0.1%
8.641481499e-05	1	0.1%
Other values (209)	209	25.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	609	73.6%
1.787748248e-08	1	0.1%
7.533239162e-08	1	0.1%
8.422593326e-08	1	0.1%
1.129931644e-07	1	0.1%

Value	Count	Frequency (%)
0.1141962336	1	0.1%
0.1098324339	1	0.1%
0.1052694858	1	0.1%
0.1006453523	1	0.1%
0.09470697174	1	0.1%

nm_pct
Real number (ℝ_≥0)

HIGH CORRELATION
ZEROS

Distinct	432
Distinct (%)	52.2%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	0.06130904045
Minimum	0
Maximum	2.737446148
Zeros	396
Zeros (%)	47.9%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	3.853871966e-05
Q3	0.01433806525
95-th percentile	0.2805295462
Maximum	2.737446148
Range	2.737446148
Interquartile range (IQR)	0.01433806525

Descriptive statistics

Standard deviation	0.2460776768
Coefficient of variation (CV)	4.013725789
Kurtosis	63.47212161
Mean	0.06130904045
Median Absolute Deviation (MAD)	3.853871966e-05
Skewness	7.330102662
Sum	50.70257646
Variance	0.060554223
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	396	47.9%
0.0007242563953	1	0.1%
0.0009563861411	1	0.1%
5.52602057e-05	1	0.1%
0.03304927327	1	0.1%
0.04616973831	1	0.1%
0.1425376239	1	0.1%
0.01916801369	1	0.1%
0.08091593375	1	0.1%
0.001924911933	1	0.1%
Other values (422)	422	51.0%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	396	47.9%
3.16228032e-07	1	0.1%
2.673417388e-06	1	0.1%
3.142061705e-06	1	0.1%
5.819624304e-06	1	0.1%

Value	Count	Frequency (%)
2.737446148	1	0.1%
2.648634738	1	0.1%
2.646402706	1	0.1%
2.166459918	1	0.1%
1.956577907	1	0.1%

ee_pct
Real number (ℝ_≥0)

ZEROS

Distinct	793
Distinct (%)	95.9%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	1.074182849
Minimum	0
Maximum	13.43708864
Zeros	35
Zeros (%)	4.2%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0.0004236764614
Q1	0.09319228164
median	0.5734127815
Q3	1.268505611
95-th percentile	4.121462343
Maximum	13.43708864
Range	13.43708864
Interquartile range (IQR)	1.175313329

Descriptive statistics

Standard deviation	1.732035158
Coefficient of variation (CV)	1.612421162
Kurtosis	15.85436852
Mean	1.074182849
Median Absolute Deviation (MAD)	0.5289223117
Skewness	3.585374758
Sum	888.3492162
Variance	2.999945789
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	35	4.2%
1.813386383	1	0.1%
0.04397889795	1	0.1%
1.134184636	1	0.1%
0.7265920897	1	0.1%
1.587616897	1	0.1%
3.375813451	1	0.1%
1.317635684	1	0.1%
0.0009877495609	1	0.1%
2.236405442	1	0.1%
Other values (783)	783	94.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	35	4.2%
1.333502244e-06	1	0.1%
3.820001528e-05	1	0.1%
0.0001033742029	1	0.1%
0.0002552959094	1	0.1%

Value	Count	Frequency (%)
13.43708864	1	0.1%
12.63353894	1	0.1%
12.36571168	1	0.1%
11.9080022	1	0.1%
11.1140622	1	0.1%

dem_res_cust_pct
Real number (ℝ_≥0)

ZEROS

Distinct	803
Distinct (%)	97.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	14.14745107
Minimum	0
Maximum	99.99625741
Zeros	25
Zeros (%)	3.0%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0.001321281614
Q1	0.2068919757
median	7.02880358
Q3	22.90882002
95-th percentile	48.69266018
Maximum	99.99625741
Range	99.99625741
Interquartile range (IQR)	22.70192805

Descriptive statistics

Standard deviation	18.45934372
Coefficient of variation (CV)	1.304782298
Kurtosis	4.208027365
Mean	14.14745107
Median Absolute Deviation (MAD)	7.017712508
Skewness	1.868540685
Sum	11699.94204
Variance	340.7473707
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	25	3.0%
0.007985554133	1	0.1%
47.04712362	1	0.1%
7.611402502	1	0.1%
0.1637679101	1	0.1%
1.484907023	1	0.1%
3.55934817	1	0.1%
0.7423241353	1	0.1%
0.005756770682	1	0.1%
8.111551312	1	0.1%
Other values (793)	793	95.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	25	3.0%
0.0002958515693	1	0.1%
0.0004810308904	1	0.1%
0.0004931425013	1	0.1%
0.0004953258926	1	0.1%

Value	Count	Frequency (%)
99.99625741	1	0.1%
99.99613825	1	0.1%
99.99601435	1	0.1%
99.98140591	1	0.1%
99.97924409	1	0.1%

circuit_pct
Real number (ℝ_≥0)

ZEROS

Distinct	232
Distinct (%)	28.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%

Mean	25.3615233
Minimum	0
Maximum	100
Zeros	451
Zeros (%)	54.5%
Memory size	6.5 KiB

Quantile statistics

Minimum	0
5-th percentile	0
Q1	0
median	0
Q3	55
95-th percentile	100
Maximum	100
Range	100
Interquartile range (IQR)	55

Descriptive statistics

Standard deviation	37.81859357
Coefficient of variation (CV)	1.491179892
Kurtosis	-0.6091574692
Mean	25.3615233
Median Absolute Deviation (MAD)	0
Skewness	1.056749176
Sum	20973.97977
Variance	1430.24602
Monotocity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0	451	54.5%
100	77	9.3%
1.666666667	7	0.8%
25	6	0.7%
50	4	0.5%
93.25842697	4	0.5%
74.41860465	4	0.5%
56.41025641	4	0.5%
60	3	0.4%
38.55421687	3	0.4%
Other values (222)	264	31.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0	451	54.5%
0.1537785589	1	0.1%
0.154218991	1	0.1%
0.2352941176	2	0.2%
0.2370532458	2	0.2%

Value	Count	Frequency (%)
100	77	9.3%
99.45945946	1	0.1%
98.80478088	1	0.1%
98.44155844	1	0.1%
97.4137931	1	0.1%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

First rows

	year	name	state	total_mwh	total_cust	no	type	saidi_nomed	saifi_nomed	caidi_nomed	circuits	voltage	nerc	gen_mwh	purchase_mwh	pv_mwh	wind_mwh	nm_mwh	ee_mwh	dem_res_customers	dem_res_mwh	pv_pct	wind_pct	nm_pct	ee_pct	dem_res_cust_pct	circuit_pct
0	2019.0	Allamakee-Clayton El Coop, Inc	IA	133395.0	9987.0	329.0	Cooperative	141.600	0.965	146.73575	53.0	0.0	MRO	148.0	150540.0	1035.749	0.144	1035.893	1124.000	2768.0	0.0	0.776453	0.000108	0.776561	0.842610	27.716031	0.000000
1	2019.0	City of Ames - (IA)	IA	639455.0	26956.0	554.0	Municipal	15.790	0.19	83.105263	41.0	0.0	MRO	126113.0	663514.0	0.000	0.000	0.000	4218.000	10683.0	0.0	0.000000	0.000000	0.000000	0.659624	39.631251	0.000000
2	2019.0	City of Anaheim - (CA)	CA	2237118.0	120279.0	590.0	Municipal	42.120	0.57	73.894737	123.0	123.0	WECC	22353.0	3009839.0	0.000	0.000	0.000	26739.841	14.0	0.0	0.000000	0.000000	0.000000	1.195281	0.011640	100.000000
3	2019.0	Arizona Public Service Co	AZ	27844577.0	1260386.0	803.0	Investor Owned	86.800	0.843	102.9656	1386.0	0.0	WECC	26805008.0	7156889.0	0.000	0.000	0.000	143225.000	693725.0	977215.0	0.000000	0.000000	0.000000	0.514373	55.040678	0.000000
4	2019.0	Entergy Arkansas LLC	AR	21818109.0	713072.0	814.0	Investor Owned	308.800	1.72	179.53488	1255.0	0.0	SERC	27576017.0	4700126.0	0.000	0.000	0.000	248663.000	23369.0	0.0	0.000000	0.000000	0.000000	1.139709	3.277229	0.000000
5	2019.0	Atlantic City Electric Co	NJ	4889951.0	486628.0	963.0	Investor Owned	76.000	0.89	85.393258	355.0	0.0	RFC	0.0	6521522.0	0.000	0.000	0.000	13574.000	43558.0	612.0	0.000000	0.000000	0.000000	0.277590	8.950985	0.000000
6	2019.0	City of Austin - (MN)	MN	335157.0	11746.0	1009.0	Municipal	31.885	0.231	138.0303	26.0	0.0	MRO	0.0	341870.0	25.037	0.000	25.037	5596.206	6809.0	140.882	0.007470	0.000000	0.007470	1.669727	57.968670	0.000000
7	2019.0	Austin Energy	TX	13695862.0	499542.0	1015.0	Municipal	68.680	0.87	78.942529	404.0	75.0	TRE	9115458.0	5666762.0	0.000	0.000	0.000	121336.320	68343.0	86.92	0.000000	0.000000	0.000000	0.885934	13.681132	18.564356
8	2019.0	Baltimore Gas & Electric Co	MD	12506681.0	965643.0	1167.0	Investor Owned	81.781	0.84	97.358333	1566.0	628.0	RFC	0.0	14754342.0	0.000	0.000	0.000	459516.583	758207.0	2871.225	0.000000	0.000000	0.000000	3.674169	78.518355	40.102171
9	2019.0	Barron Electric Coop	WI	332559.0	18850.0	1251.0	Cooperative	144.720	1.07	135.25234	37.0	0.0	MRO	122.0	352789.0	0.000	0.000	0.000	1144.000	6371.0	0.0	0.000000	0.000000	0.000000	0.343999	33.798408	0.000000

Last rows

	year	name	state	total_mwh	total_cust	no	type	saidi_nomed	saifi_nomed	caidi_nomed	circuits	voltage	nerc	gen_mwh	purchase_mwh	pv_mwh	wind_mwh	nm_mwh	ee_mwh	dem_res_customers	dem_res_mwh	pv_pct	wind_pct	nm_pct	ee_pct	dem_res_cust_pct	circuit_pct
817	2013.0	Wake Electric Membership Corp	NC	674269.0	36896.0	19981.0	Cooperative	83.000	1.304	63.650307	73.0	0.0	SERC	.	709377.0	45.145	0.000	45.145	2206.0	1.0	0	0.006695	0.000000e+00	0.006695	0.327169	0.002710	0.000000
818	2013.0	City of Waseca - (MN)	MN	61010.0	4148.0	20136.0	Municipal	30.000	0.77	38.961039	5.0	NaN	MRO	.	64314.0	0.000	0.000	0.000	445.0	1601.0	35	0.000000	0.000000e+00	0.000000	0.729389	38.596914	0.000000
819	2013.0	West Penn Power Company	PA	7494953.0	505069.0	20387.0	Investor Owned	151.000	1.09	138.53211	830.0	830.0	RFC	.	7998118.0	0.000	0.000	0.000	212827.0	0.0	.	0.000000	0.000000e+00	0.000000	2.839604	0.000000	100.000000
820	2013.0	Western Massachusetts Electric Company	MA	1822410.0	187444.0	20455.0	Investor Owned	88.000	1	88	219.0	0.0	NPCC	5083	1944297.0	0.000	0.000	0.000	110826.0	1.0	2	0.000000	0.000000e+00	0.000000	6.081288	0.000533	0.000000
821	2013.0	Withlacoochee River Elec Coop	FL	3565155.0	202352.0	20885.0	Cooperative	115.000	1.621	70.943862	181.0	0.0	FRCC	.	3720951.0	0.000	0.000	0.000	38.0	202310.0	1532	0.000000	0.000000e+00	0.000000	0.001066	99.979244	0.000000
822	2013.0	Westar Energy Inc	KS	9826375.0	373094.0	22500.0	Investor Owned	101.930	1.244	81.937299	905.0	NaN	SPP	15175161	3283074.0	0.000	0.000	0.000	885.0	29825.0	.	0.000000	0.000000e+00	0.000000	0.009006	7.993964	0.000000
823	2013.0	Tucson Electric Power Co	AZ	9278919.0	409529.0	24211.0	Investor Owned	63.499	0.683	92.970717	420.0	0.0	WECC	11311182	2659239.0	48733.701	0.080	48733.781	138815.0	37.0	.	0.525209	8.621694e-07	0.525210	1.496026	0.009035	0.000000
824	2013.0	Cass County Elec Coop Inc	ND	1119192.0	39897.0	24949.0	Cooperative	46.000	1.11	41.441441	39.0	10.0	MRO	.	1158278.0	65.129	60.513	125.642	34.0	9985.0	18380	0.005819	5.406847e-03	0.011226	0.003038	25.026944	25.641026
825	2013.0	Western Indiana Energy REMC	IN	628689.0	16603.0	25295.0	Cooperative	238.010	.	.	68.0	0.0	RFC	.	650274.0	14.400	0.000	14.400	1214.0	350.0	7288	0.002290	0.000000e+00	0.002290	0.193100	2.108053	0.000000
826	2013.0	Entergy Texas Inc.	TX	16813590.0	421105.0	55937.0	Investor Owned	144.900	1.32	109.77273	479.0	0.0	SERC	7033780	17919380.0	0.000	0.000	0.000	30979.0	8.0	5966	0.000000	0.000000e+00	0.000000	0.184250	0.001900	0.000000

Overview

Variables

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows